Computer implemented system and method of identification of useful untested states of an electronic design

ABSTRACT

A computer implemented system and method of computer implemented method of instrumentation of an electronic design comprising receiving by a computer a computer readable representation of said electronic design having at least in one part of said electronic design, an analog portion. At least one instrumented netlist is generated based at least in part upon said representation of said electronic design. At least one specification of said electronic design is received and at least one set of valid states is generated based on said at least one specification. An analog verification coverage is determined utilizing said at least one instrumented netlist.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Divisional of U.S. patent application Ser. No.14/707,723, filed on May 8, 2015, and entitled COMPUTER IMPLEMENTEDSYSTEM AND METHOD OF IDENTIFICATION OF USEFUL UNTESTED STATES OF ANELECTRONIC DESIGN, now U.S. Pat. No. 9,875,325 issuing Jan. 23, 2018(Atty. Dkt. No. ZPLG-32170). U.S. patent application Ser. No. 14/707,723claims benefit of U.S. Provisional Application No. 61/991,069, filed onMay 9, 2014, entitled COMPUTER IMPLEMENTED SYSTEM AND METHOD OFIDENTIFICATION OF USEFUL UNTESTED STATES OF AN ELECTRONIC DESIGN (Atty.Dkt. No. ZPLG-32197). U.S. application Ser. Nos. 14/707,723 and61/991,069 and U.S. Pat. No. 9,875,325 are incorporated herein byreference in their entireties.

BACKGROUND

The method and system are generally related to the verification ofanalog and mixed signal integrated circuits and is particularly usefulin, but not limited to instrumenting an electronic design to assesscompleteness of verification and identify useful untested states.

Electronic design automation (EDA) is software for designing electronicblocks. There are several broad types of electronic signals, componentsand blocks: digital, analog and a mixture of digital and analog termedmixed signal. The electronic design generally comprises at least one ofthe following levels of circuit information: a system level, anarchitectural level, a dataflow level, an electrical level, a devicelevel and a technology level and/or the like.

Digital signals have discrete input and output values “0” and “1”,occurring at discrete time values, typically tied to a clock signal.Digital components which input and output the digital signals typicallyhave static pin outs and interaction protocols. Digital blocks comprisedof the digital components have well established and well documentedphysical layouts and electrical interactions. Simulators for digitalblocks are discrete time event driven simulators.

Analog signals generally have continuous input and output values thatmay vary over time. Analog components typically have customizablelayouts in order to modify inputs, outputs, triggers, biases, etc.Therefore, due to customization, analog blocks comprised of the analogcomponents may not have as well established or well documented physicallayouts or electrical interactions as digital circuits. Simulators foranalog blocks generally necessitate continuous time domain simulators.

Mixed signal blocks are a combination of digital signal blocks andanalog signal blocks within a component being simulated. The most commonoptions available for simulation are to simulate the component as agrouping of analog blocks, or to separately analyze the analogcomponents/blocks and the digital components/blocks and translate theinputs and outputs at the boundaries of the digital and analog domainsfor inter-domain communication.

Within EDA there are two broad categories of circuit review that areoften related: simulation and verification. Simulation is a numericalsolution set that predicts the behavior of a circuit. Verification isthe systematic pursuit of describing the behavior of a circuit underrelevant conditions (functional verification) and over manufacturingprocess variation (parametric verification). Therefore, verificationgenerally necessitates a much more extensive review of the circuit, itsoperating conditions, and manufacturing operation variations than asimulation. It is possible to run a large number of simulations withoutverifying to any significant degree the functionality of a circuit.Verification is the mathematical modeling of circuit behavior andevaluation of circuit performance over a range of conditions.Ultimately, the measure of success of verification is to report how wellthe circuit design complies with the circuit specification. Analog andmixed signal verification methodology is struggling to keep pace withthe complexity, cost, and computational demands of ever-growing analogand mixed signal circuits.

The number and complexity of verification test cases grows with thecomplexity of analog and mixed signal designs. Additionally, simulationspeed decreases and memory utilization increases as the size of thecircuit grows. Thus, the computational processing-power to verify acircuit may dramatically increase with circuit complexity. To make thisissue more painful, verification normally occurs at the end of a designcycle where schedule delays are perceived to be most severe. Thus,verification is an activity that generally necessitates a significantamount of simulation processing-power for a small part of the overalldesign cycle, and therefore an efficient use of verification resourcesis generally necessitated to meet time to market demands.

Today's complex verification solutions specifically focus engineering onthe verification activity to ensure that the operation of the circuit isfully and efficiently verified under pertinent conditions. This focusedanalog and mixed signal verification is much more manual and experiencedriven than digital verification. This sporadic interactive analogverification leaves companies at risk. The present disclosure may allowverification tasks to be defined at a higher level of abstraction. Thepresent disclosure may allow efficient capture of complex relationshipsbetween stimulus or stimulus assertions and output measurements oroutput assertions. The present disclosure may allow the test oftransistor level circuits, circuits implemented with behavioral models,or circuits that contain a combination of behavioral models andtransistor level implementations. The methods used presently formodeling analog and mixed signal circuits are not efficient forminimizing the number of verification runs to exercise the valid states.This is due at least in part to the fact that the netlist isinsufficiently instrumented to efficiently record the states exercised.A netlist describes the connectivity of the electronic design. There isa long felt need for instrumenting a netlist to identify valid usefuluntested states of the electronic design.

Robust verification of analog and mixed signal circuits generallynecessitates a significant investment in test benches, performanceanalysis routines, and macro-models that may be used to accelerate thesimulations. The complexity of this collateral grows with the complexityof the analog and mixed signal integrated circuits to be verified. As adesign team adds design resources it also needs to add verificationresources, adding to the cost of the design. The efficient use of thoseresources becomes paramount due to the inevitable time constraints thatare imposed at the end of the design cycle, when companies are trying toget a product to market.

The current technology trajectory, within the electronics manufacturingindustry, is to move more and more toward single chip designs, calledSystems on a Chip (SoC), or multi-chip modules (MCM) where multiplechips are included in one package. Most systems on a chip and multi-chipmodules generally necessitate some level of mixed signal verification.As mixed signal designs continue to increase in size and complexity,this places additional burdens on verification to insure first passdesign success and reduce the time-to-market. Although the complexity ofanalog and mixed signal ASIC design has followed Moore's law,innovations in design verification generally have not.

Valuable design time and computational resources as well as expensivesimulator resources may be specifically focused by the disclosed methodfor instrumenting the netlist in order to ascertain the minimumverification runs necessary to test valid states. The method makesassessment of global verification more efficient. The resultingminimized run list allows more efficient use of resources.

This disclosure is related to instrumenting a netlist of an electronicdesign for analog and mixed signal (A/MS) application specificintegrated circuits (ASICs). Analog and mixed signal integrated circuitsexist in many modern electronic devices, and these circuits needs to beverified through simulation prior to fabrication.

SUMMARY

There is provided according to, just one exemplary embodiment and it'saspect of the present disclosure of a computer implemented system andmethod of instrumentation of an electronic design comprising receivingby a computer a computer readable representation of said electronicdesign having at least in one part of said electronic design, an analogportion. At least one instrumented netlist is generated based at leastin part upon said representation of said electronic design. At least onespecification of said electronic design is received and at least one setof valid states is generated based on said at least one specification.An analog verification coverage is determined utilizing said at leastone instrumented netlist.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure will be more clearly understood fromconsideration of the following detailed description and drawings inwhich:

FIG. 1 is a block diagram showing a computer system suitable forpracticing the instant disclosure;

FIG. 2 is a block diagram showing a computer network system suitable forpracticing the instant disclosure;

FIG. 3 depicts an example Low Voltage Dropout (LDO) circuit;

FIG. 4 depicts an example amplifier circuit;

FIG. 5 depicts a test bench pin out for an amplifier;

FIG. 6 depicts a general example hierarchy;

FIG. 7 depicts an instance parsed example test hierarchy;

FIG. 8 depicts a first example test bench for a power managementintegrated circuit;

FIG. 9 depicts a second example test bench for a power managementintegrated circuit;

FIG. 10 depicts a third example test bench for a power managementintegrated circuit;

FIG. 11 depicts an example simple MOSFET current sink;

FIG. 12 depicts an example cascade MOSFET current sink;

FIG. 13 depicts an example mux and opamp of an electronic design;

FIG. 14 depicts a B level example of an electronic design;

FIG. 15 depicts a D level example of an electronic design;

FIG. 16 depicts a hierarchical example of an electronic design;

FIG. 17 depicts an E level example of an electronic design;

FIG. 18 depicts a first example of a computer implemented method ofidentification of useful untested states of an electronic design;

FIG. 19 depicts additional steps that the first example ofidentification of useful untested states of an electronic design mayadditionally comprise;

FIG. 20 depicts additional steps that the first example ofidentification of useful untested states of an electronic design mayadditionally comprise;

FIG. 21 depicts a second example of a computer implemented method ofidentification of useful untested states of an electronic design;

FIG. 22 depicts additional steps that the second example ofidentification of useful untested states of an electronic design mayadditionally comprise;

FIG. 23 depicts a third example of a computer implemented method ofidentification of useful untested states of an electronic design;

FIG. 24 depicts additional steps that the third example ofidentification of useful untested states of an electronic design mayadditionally comprise;

FIG. 25 depicts a fourth example of a computer implemented method ofidentification of useful untested states of an electronic design;

FIG. 26 depicts additional steps that the fourth example ofinstrumentation of identification of useful untested states of anelectronic design;

FIG. 27 depicts a fifth example of a computer program product embodiedon a non-transitory computer usable medium to identify of usefuluntested states of an electronic design;

FIG. 28 depicts additional steps that the fifth example of a computerprogram product embodied on a non-transitory computer usable medium toidentification of useful untested states of an electronic design mayadditionally comprise;

FIG. 29 depicts a sixth example of a computer-based system ofidentification of useful untested states of an electronic design; and

FIG. 30 depicts an example system diagram for identification of usefuluntested states of an electronic design.

References in the detailed description correspond to like references inthe various drawings unless otherwise noted. Descriptive and directionalterms used in the written description such as right, left, back, top,bottom, upper, side, et cetera, refer to the drawings themselves as laidout on the paper and not to physical limitations of the disclosureunless specifically noted. The drawings are not to scale, and somefeatures of examples shown and discussed are simplified or amplified forillustrating principles and features as well as advantages of thedisclosure.

DETAILED DESCRIPTION

The features and other details of the disclosure will now be moreparticularly described with reference to the accompanying drawings, inwhich various illustrative examples of the disclosed subject matter areshown and/or described. It will be understood that particular examplesdescribed herein are shown by way of illustration and not as limitationsof the disclosure. Furthermore, the disclosed subject matter should notbe construed as limited to any of examples set forth herein. Theseexamples are provided so that this disclosure will be thorough andcomplete, and will fully convey the scope of the disclosed subjectmatter to those skilled in the art. The principle features of thisdisclosure may be employed in various examples without departing fromthe scope of the disclosure.

The terminology used herein is for the purpose of describing particularexamples only and is not intended to be limiting of the disclosedsubject matter. Like number refer to like elements throughout. As usedherein the term “and/or” includes any and all combinations of one ormore of the associated listed items. Also, as used herein, the singularforms “a”, “an”, and “the” are intended to include the plural forms aswell, unless the context clearly indicates otherwise. It will be furtherunderstood that the terms “comprises”, and/or “comprising” when used inthis specification, specify the presence of stated features, integers,steps, operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or groups thereof. Also, as usedherein, relational terms such as first and second, top and bottom, leftand right, and the like may be used solely to distinguish one entity oraction from another entity or action without necessarily requiring orimplying any actual such relationship or order between such entities oractions.

Cost of entry barriers into analog and mixed signal IC design is endemicespecially to fabless companies that are developing ASIC intellectualproperty in the form of packaged ASICs or modules to be integrated intotheir customer's Systems-on-Chip (SoCs) or multi-chip modules (MCMs).For example, if a fabless design center is staffed with five IC designengineers, equipping the team with design tools is financiallyequivalent to quadrupling the staff. This is due to the high cost ofownership of the EDA tools, not just in annual license fees,installation and support, training and the like. Reducing system usethrough instrumentation of the netlist to capture valid stateverification data allows more efficient resource allocation.

Analog and mixed signal verification is time and computationallyintensive. Functionality of the circuit for various inputs, at variousconditions and for various manufacturing conditions are generallynecessitated to be simulated to insure that the circuit functions to thespecifications.

Prior to running a simulation of an electronic design, the electronicdesign undergoes a conversion to a netlist which describes theconnectivity of the electronic design. The netlist whilst describing theconnectivity of the circuit does not include metadata pertaining to thecircuit.

The disclosed system and method of instrumentation of an electronicdesign instruments the netlist to capture metadata pertaining to theverification. This captured metadata in conjunction with generation ofvalid states from the specification allows an assessment of thecompletion of the verification of the electronic design.

AMST™ (Analog Mixed Signal Test) is a module for specifyinganalog/mixed-signal (A/MS) stimulus as well as assertions and outputmeasurements. An AMST is able to efficiently capture complexrelationships between stimulus and output assertions. Verificationmodels specified in the AMST Language (AMSTL™) captures higher levelcommands that are subsequently translated into Verilog-A/AMS. Verilog-A,Verilog-AMS, VHDL-AMS, SystemC-AMS or the like, which are standardizedlanguages for defining analog and mixed-signal, respectively. It isenvisioned that the code could also be used to generate any languagestandard that supports direct branch contribution statements for analogsimulators.

AMSTL can be used to capture higher level commands with IBCS regardlessof whether the resulting translated code will be used in the behavioralmodel of a circuit or a test harness. The value of capturing assertionswithin an AMST rather than in the behavioral model of the circuit isthat the verification commands can be reused regardless of therepresentation of the circuit. For example, in FIG. 17 the AMP_AMSTblock will execute the same commands and assertions whether the op ampand mux are represented as transistor level schematics or behavioralmodels. The AMP_AMST now serves as verification intellectual property(VIP) that can be reused with these circuits. This concept becomesespecially valuable when considering the case of providing analog designintellectual property (IP) to a third party. The purchasing party nowhas verification IP that can be embedded with the purchased IP within alarger SOC. This VIP reduces the risk that the purchased design IP willbe used incorrectly. Verification IP has been a proven concept fordigital circuits and top-level inputs and outputs but has not beenpreviously practical to provide with embedded analog IP.

The language AMSTL is intended to describe behaviors, stimulus, outputs,measurements, etc. for analog mixed signal integrated circuit design andprovides higher level constructs than available in standard hardwaredescription languages that are intended for input into analogsimulators. One benefit is an efficiency improvement based on theavailability of IBCS in AMSTL. Additionally, AMSTL code can be parsed tooutput any desired standard language. The module AMST is a behavioralmodel of analog mixed-signal verification intellectual property and isintended to reside at any level of the hierarchy within the design. Themodule may reside with the IP that it is monitoring, stimulating and/orevaluating. The output form of the model (in Verilog-A, Verilog-AMS,VHDL-AMS, etc.) can be input through a netlist into an analog ormixed-signal simulator.

A netlist is a representation of one or more databases that containinformation relevant to a verification project and simulation taskincluding but not limited to:

-   -   1) a description of components (e.g. transistors, resistors,        capacitors, behavioral model, AMSTs, digital gates) and        properties of components that make up the design (e.g. PMOS2 has        W=1 um),    -   2) the connectivity of the design (e.g. drain of PMOS1 is        connection to gate of NMOS2),    -   3) hierarchical configuration of the design for a specific        simulation task (e.g. PLL1 is represented as a model, LDO3 is        represented at the transistor level)    -   4) configuration of the simulation task including simulation        type (e.g. transient simulation), duration (e.g. 2 ms),        tolerance settings (e.g. iabstol<10e−10), configuration of        interface elements between digital and analog partitions, and        output signal selections    -   5) any information in the verification database such as expected        performance values, signal transitions, signal shape, duty        cycle, etc of any signal or element of interest to the        verification activity).

The netlist is the input to a simulation. The purpose of a simulation isto predict the behavior of the circuit described in the netlist subjectto the stimulus conditions and accuracy criteria specified in thenetlist. Simulation is an essential part of integrated circuit (IC)design since a) photomasks for IC designs are very expensive b) ICmanufacturing takes a long time, c) probing of signals internal to an ICis extremely difficult and d) bread-boarding of modern IC designs isimpractical. A simulation is performed with a simulator. At a highlevel, there are three approaches to simulating an integrated circuit:SPICE-level simulation, digital-level simulation, and mixed-modesimulation. A SPICE-level simulator reduces the netlist to a set ofnonlinear differential algebraic equations which are solved usingimplicit integration methods, Newton's method, and sparse matrixtechniques. A SPICE-level simulator conserves charge, satisfiesKirchhoff's Current Law and Kirchhoff's Voltage Law, subject to a set ofabsolute or relative tolerances. A digital simulator reduces the netlistto a set of boolean functions which are triggered by discrete events.Digital simulators do not conserve charge, satisfy Kirchhoff's CurrentLaw or Kirchhoff's Voltage Law. But they can simulate much largercircuits at a higher level of abstraction. Mixed-mode (AMS) simulationcombines a SPICE-level simulator with a digital simulator. In this typeof simulation a SPICE-level simulator is used to simulate a portion ofthe design, predicting the net voltages and terminal currents of thecomponents in the SPICE-level partition, while the digital simulator isused to predict the digital outputs of the components in the digitalpartition. In a mixed-mode simulation, the SPICE-level partition and thedigital partition are connected with interface elements which, at abasic level, are idealized 1-bit analog to digital converters (forsignals going from the SPICE partition to the digital partition) and1-bit digital to analog converters (for signals going from the digitalpartition to the SPICE partition).

A simulation can produce the following outputs:

-   -   1) continuous-time/continuous-value waveforms of net voltages        and terminal currents    -   2) discrete-time/discrete-value digital waveforms of logic net        outputs    -   3) any data written by any behavioral model including any AMST        modules that have been included into the netlist    -   4) assertion violation messages    -   5) debug information about model behavior, circuit convergence        difficulties, etc.

These outputs from the simulation are stored in one or more databases.These outputs are subsequently used to evaluate the suitability of thecircuit. This process can be manual. A designer can, for example, reviewwaveforms in a graphical waveform viewer. The process can also beautomated. A software program can programmatically analyze waveformresults and AMST outputs to build a spec compliance matrix whichsummarizes the set of design objectives that have been satisfied and theset of design objectives that have been failed in the circuitsimulation.

An instrumentation point may be:

-   -   1) any component in the netlist (e.g. transistors, resistors,        capacitors, behavioral model), or    -   2) any net that defines some aspect of the connectivity of the        design, or    -   3) any arbitrary set of one or more components and zero or more        nets, or    -   4) any arbitrary set of zero or more components and one or more        nets.

Instrumentation points can be created in one of three ways:

-   -   1) Manually specified by the user of the system. In this usage        scenario, the user manually identifies instrumentation points.        This can be accomplished from the user's schematic capture        environment. As a simple example, the user could select two nets        as an instrumentation point. The instrumentation module, called        an AMST, can be implemented in Verilog-A/AMS or any other        hardware description language. In the instrumentation module the        user can specify the shape, behavior, or transfer function        between the two nets that identifies a desirable or undesirable        mode or operation. A more complex example, consisting of many        nets, is shown in FIG. 17. Once specified, the software can        track coverage of these desirable or undesirable behaviors.    -   2) Programmatically identified from programmatically described        patterns. Analog and mixed-signal circuits frequently have        repeating topological patterns such as the current mirror        consisting of M1 and M2 in FIG. 11. In this topology, the        current through M2 is a fixed ratio of I_REF if certain        conditions are satisfied. These topologies can be        programmatically identified, assertions or AMSTs for these        topologies can be automatically generated and these assertions        or AMSTs can be automatically instrumented to measure coverage        for the topological pattern. Static patterns for identifying        circuit function can incorporate circuit topology (specific        connections between specific device terminals), device names and        types, device properties (such as device model names), net names        and net properties (such as net width).    -   3) Programmatically derived from aggregated analytic        information. Static analysis of netlist constructs is limited as        a mechanism for deriving circuit function as the same circuit        topology can used for different applications (with different        biasing or stimulus). To overcome this problem, additional        instrumentation points can be derived from historical analytic        information such as known-good simulation results. For example,        a program can analyze the simulation results from the current        mirror in FIG. 12, determine that the current flowing from        source to drain of M4 is in a certain range, automatically        generate assertions or AMSTs for this particular instance of the        circuit in a larger design. These assertions or AMSTs can then        be automatically instrumented to measure coverage for the        topological patterns.

Instrumentation can be accomplished in two different ways:

-   -   1) Instrumentation of Existing Behavioral Models and AMSTs.        Behavioral model and hand-written AMSTs are very useful because        they capture a mathematical description of the desired function        of the circuit. Behavioral models implement the desired function        directly. AMSTs describe circuit function indirectly through a        set of assertions that test circuit response to a specified        stimulus. Both can be instrumented for coverage by a) tracking        the execution of conditional branch statements in the model        and/or b) dividing any transfer function implemented in the        model into distinct regions and tracking the use of each region.        A simple example of a conditional branch statement is an        if-then-else clause in Verilog-A/AMS. To measure coverage, the        software tracks how many of the conditional branches in each        instance of each behavioral model are executed in the        verification simulation. The transfer function resulting from        the model can also be analyzed to identify any discontinuities        in the transfer function, its first, second, third, fourth, or        arbitrary n-th derivative. Any identified discontinuity is used        to partition the model into regions of operation. To measure        coverage, the software tracks the number of regions of operation        reached for each instance of each model in the verification        simulation.    -   2) Automatic Insertion of Instrumented AMSTs. For        transistor-level circuits the desired function is not always        known. As described above, an AMST module for an instrumentation        point can be automatically generated from static analysis of the        netlist or aggregated analytic information that can incorporate        known-good simulation results, simulation configuration, user        inputs, or observed user behavior. Once the module is generated,        it can be added into the netlist by adding an instance of the        new module in the appropriate part of the netlist, connecting        voltage measurement nets, and splicing any nets where current        measurement is required. Once connected, the AMST can be        instrumented as described above.

Therefore the disclosed system and method of translation ofinstrumentation of an electronic design may solve one or more of thefollowing issues, to allow more efficient use of computer and personnelresources through reduced verification run overlaps, to reduce the timelag to market and/or to insure a more focused and thorough verificationconfirmation.

Computer System FIG. 1 illustrates the system architecture, for anexemplary computer system 100, on which the current disclosure may beimplemented. The exemplary computer system of FIG. 1 is for descriptivepurposes only. Although the description may refer to terms commonly usedin describing particular computer systems, such as a personal computer,the description and concepts equally apply to other systems, includingsystems having architectures dissimilar to FIG. 1.

Computer system 100 typically includes a central processing unit (CPU)110, which may be implemented with one or more microprocessors, a randomaccess memory (RAM) 112 for temporary storage of information, and a readonly memory (ROM) 114 for permanent storage of information. A memorycontroller 116 is provided for controlling RAM. A bus 118 interconnectsthe components of the computer system. A bus controller 120 is providedfor controlling the bus. An interrupt controller 122 is used forreceiving and processing various interrupt signals from the systemcomponents. Mass storage may be provided by flash 124, DVD 126, or harddisk 128, or, for example a solid-state drive. Data and software may beexchanged with the computer system via removable media such as the flashdrive and DVD. The flash drive is insertable into a Universal SerialBus, USB, drive 130, which is, in turn, connected to the bus by acontroller 132. Similarly, the DVD is insertable into DVD drive 134,which is, in turn, connected to bus by controller 136. Hard disk is partof a fixed disk drive 138, which is connected to the bus by controller140.

User input to the computer system may be provided by a number ofdevices. For example, a keyboard 142 and a mouse 144 are connected tothe bus by a controller 146. An audio transducer 148, which may act as amicrophone and a speaker, is connected to bus by audio controller 150,as illustrated. Other input devices, such as a pen and/or tabloid, maybe connected to the bus and an appropriate controller and software. DMAcontroller 152 is provided for performing direct memory access to thesystem RAM.

A visual display is generated by video subsystem 154, which controlsvideo display 156. The computer system also includes a communicationsadaptor 158, which allows the system to be interconnected to a localarea network (LAN) or a wide area network (WAN) or other suitablenetwork, schematically illustrated by a bus 160 and a network 162.

Operation of the computer system is generally controlled and coordinatedby an operating system, such as the Windows, Windows 7 and Windows 8operating systems, available from Microsoft Corporation, Unix, Linux orApple OS X operating system, to name a few. The operating systemcontrols allocation of system resources and performs tasks such asprocessing scheduling, memory management, networking, and I/O services,among other things.

Computer System FIG. 2 illustrates the system 200 in which the computeruser 210 is connected to a network 212 which in turn is connected to thecloud 214 and the compute farm 216.

In an example schematic of a circuit to be verified/analyzed, a lowvoltage dropout (LDO) 300 circuit is shown in FIG. 3. The LDO has anamplifier A1, having an inverting input (−input), a non-inverting input(+input) an output, a positive voltage input +V and a negative voltageinput −V. The LDO circuit has a voltage in Vin and a voltage out Vout.The LDO has a power out block Q1, Q2 and R2. The LDO feedback circuit iscomprised of R3, R4, D1 and R1. The amplifier A1 is termed a symbol, theelements D1, R1, R2, R3, R4, C1, C2, Q1 and Q2 are referred to asprimitives.

An example schematic of an amplifier A1 400 circuit is shown in FIG. 4.The symbol of the amplifier is comprised of transistors Q3, Q4, Q5, Q6,Q7 and Q8 and resistor R5. The amplifier A1, having an inverting input(−input), a non-inverting input (+input) an output, a positive voltageinput +V and a negative voltage input −V.

FIG. 5 shows a test bench 500 for amplifier A1 510. A test bench is aspecific configuration of inputs, outputs, test conditions and the likethat are run for a device to which it is connected. The test bench hasan inverting input 512, a non-inverting input 514, a positive powerinput 516, a negative power input 518 and an output 520. The test benchhas associated connections, power supplies, IOs, etc. which are referredto as the test bench collateral. The portion around the periphery of thecircuit is referred to as the verification harness. Pin outs and theoperation of the verification harness need to be matched to the circuitunder test.

FIG. 6 shows a general example hierarchy 600 of a Test bench with adevice under test, DUT. The hierarchy is arranged according to levels,A, B, C and Device and according to instances 1, 2 and 3. The connectinglines indicate which models are connected throughout the hierarchy for aspecific verification. Within level and instance, multiple view typesmay exist. The examples illustrate some possible hierarchicalconfigurations and are not intended to limit the cases and views or viewtypes.

Integrated circuit design hierarchy is the representation of integratedcircuit designs utilizing hierarchical representations. Thisrepresentation allows for more efficient creation of complex designsthat may include millions of components such as transistors, resistors,and capacitors as well as the metal lines that connect the devices. Thedesign hierarchy representation used at any given point in the designprocess may vary based on the design step being performed and the typeof design function such as analog, digital, or memory.

In the case that a design is to be manufactured, a layout of the designis created so that a representation may be mapped. This mapping allowspatterns to be created on individual levels of the mask sets to allowdesign manufacture. In general, the design flow to create the layoutrepresentation is very different for analog as compared to digitalfunctional blocks and subsystems.

Early in the design process, there may be large portions of the designthat are designed for the first time and do not have any existing layoutrepresentations. Other portions of the design may already have beenproven, and these may be represented at a higher level of abstraction orin combination may include the layout representation, or may be stockitems not even coming from the same design house.

Some common types of design representations referred to here as viewsmay comprise various view types. A Schematic view type is a picture ofcomponents or blocks with connectivity shown by lines or nets andconnections to other levels of the hierarchy through pins. A Spice viewtype is a representation of a component and its associated parameters,possibly including a specific device model that will be instantiatedinto the spice netlist. An LVSExtract is a view type that is created bya tool analyzing the layout view and reverse engineering the individualcomponents and connectivity. Variations of this type of view may alsoinclude extracted parasitic components resulting from the physicallayout that were not drawn by the designer. A Layout view type is arepresentation of the specific geometries including routing for thatportion of the design. A Verilog™ view type is a text file that is instandardized Verilog™ format. A Verilog-A™ view type is a text file instandardized Verilog-A™ syntax. A Verilog-AMS™ view type is a text filein standardized Verilog-AMS™ syntax. View type names may be differentdepending on the electronic design automation tool provider, examples ofwhich include SpectreHDL and HDL-A.

Other types of view types may help organization and readability of thehierarchy. As an example, graphic design tools such as schematic capturesystems may use a Symbol view type for the graphic that is placed. Thesymbol may contain pins that connect the instance through the hierarchyas well as a drawing that indicates the function of the block. Examplesinclude common symbols for operational amplifiers, basic digital gates,transistors, resistors, and the like.

Further adding to the complexity of description, a given block at alevel of the design hierarchy may include multiple views of the sameview type. An example would be different Verilog™ representations of agiven block, for instance, one with annotated timing based on thelayout, one with estimated timing, one without timing, or differentlevels of design representation such as gate-level or register transferlevel (RTL). Similarly, an analog view may have numerous schematicviews: for instance, one that will map to the final transistor-leveldesign, one that includes placement of behavioral blocks for higherlevel modeling, one that may include parasitic elements from the layout,one that includes interface elements between analog and digital blocksfor mixed-signal simulation. Also, for analog blocks there may bemultiple Verilog-A™ or Verilog-AMS™ model views for the same block wheremodels include different functionality and accuracy based on the purposeof different simulation exercises. These multiple views and view typesare mapped into configurations that are used for a specific task oranalysis.

Often view names are created to provide hints for what types of analysisa specific view may be useful. View name may include those listedhereinafter and the like. A Schematic is a schematic view including theplacement of blocks that may be evaluated at the transistor level or atsome level of the hierarchy such as a behavioral model. ASchematic_behavioral is a schematic view that comprises behavioralelements. A Schematic_parasitics is a schematic view that includesparasitic components extracted or estimated from the layout. A Spice isa spice view that includes the information implemented in a netlist anda component for a specific analog simulator. A Behavioral_va is a textview in the Verilog-A™ format that models a specific block for an analogsimulator that may evaluate Verilog-A™, and a Behavioral_vams is a textview in the Verilog-AMS™ format that models a specific block for amixed-signal simulator that may evaluate Verilog-A™ and Verilog.

In the specific example shown in FIG. 6, general example hierarchy, withdevice under test A1, Instance 1, would be defined based on thefollowing configuration: A1, Instance 1 and B1, Instance 1 are modeledwith a Schematic level model. B2, Instance 1 is modeled with aSchematic_behavioral model, and C1, Instance 1 and C2 Instance 1 aremodeled using a Schematic model. C1, Instance 2 and C3, Instance 1 aremodeled with a Behavioral_va model. At the bottom of the hierarchy,Devices 1, 2 and 3, instances 1, 2 and 3 are modeled using Spice.

In the specific example shown in FIG. 6, Device 1, Instance 2 is a dummydevice and therefore would not change the simulator matrix. Device 1,Instance 2 is placed in the C1, Instance 1 schematic connected as adummy device and is therefore not part of the A1, Instance 1 matrix thatwould be stamped in the simulator.

Whether a change necessitates a verification to be rerun is determinedin part by the connections through the hierarchy. In this specificexample for general example hierarchy, device under test A1, Instance 1,if Device 1, Instance 2, Schematic view is changed the simulator wouldnot need to be rerun, since the device is a dummy device and would notmodify the matrix that would be stamped into the simulator.

With a view to FIG. 6, C1, Instance 1 Schematic view forms part of theconfiguration of the simulator model, and if it is changed and thechange is substantive enough to affect the simulator matrix, Test bench1 would need to be rerun. C1, Instance 2 Schematic view would not form apart of the configuration of the simulator model example, therefore ifit is changed, Test bench 1 would not need to be rerun.

At a more abstract level, if C1, Schematic view is changed, thereforechanging the schematic view in Instance 1 and 2, which affects a changein the information stamped in the simulator matrix, Test bench 1 wouldneed to be rerun. If a non-substantive change to C1, Schematic view ismade for example by adding a comment and no change is made to theinformation stamped by the simulator in the matrix, this designconfiguration would not need to be rerun. It is apparent thatdetermining whether a change was made to a configuration and the effectof the stamping of the matrix, may have a large effect on the number ofnecessitated verification runs.

FIG. 7 shows some of the different model views that may be chosen fromfor modeling a power management chip PMIC 700. The PMIC_testbench hasBehavioral_vams and Behavioral_va levels having stimulus and oututs. ThePMIC has Schematic and Schematic_behavioral levels. The LDO, LDO EnableControl and Battery Supervisor are defined at the Schematic,Schematic_behavioral and Behavioral_vams levels. The Voltage Reference,LDO Feedback and LDO Comparator are defined at the Schematic andBehavioral_va levels. The LDO Amplifier is defined at the Schematic andSchematic_parasitics levels. The Behavioral Amplifier and BehavioralBias are defined at the Behavior_va level. The LDO Control Logic isdefined at the Schematic and Verilog™ levels, and Devices 1 through Xare defined at the Spice level.

FIG. 8 shows a first test hierarchy of the power management chip PMICdescribed in FIG. 6 for a power management chip 800. The figureillustrates a portion of the hierarchy if a Spice primitive componentconfiguration is defined. Device 1, Instance 2 is a dummy device in thismodel and would not change the simulator matrix.

FIG. 9 shows a second test hierarchy of the power management chip PMICdescribed in FIG. 7 for a power management chip 900. The figureillustrates a portion of the hierarchy for one possible mixedconfiguration with some analog behavioral level models, some Verilog™representations and some Spice primitive components.

FIG. 10 shows a third test hierarchy of the power management chip PMICdescribed in FIG. 7 for a power management chip 1000. The figureillustrates a portion of the hierarchy if a behavioral configuration isdefined.

Analog and mixed-signal circuits frequently have repeating topologicalpatterns. FIGS. 11 and 12 are examples of common easily identifiableanalog topologies. Current mirrors, also referred to as current sourcesand current sinks depending on the usage, are important tools in analogintegrated circuit design to provide matched or ratioed currentsthroughout the circuit and therefore provide precision operation. FIG.11 depicts an example simple current MOSFET current mirror. In thistopology, the current I_OUT is approximately a fixed ratio of I_REF andindependent of voltage if certain biasing conditions are satisfied.

FIG. 12 depicts an example cascode current sink. The cascode currentmirror configuration creates higher output resistance and thereforeoffers some performance advantages compared to the simpler currentsource topology shown in FIG. 11. There are numerous other basic analogbuilding block topologies in addition to these two example currentmirrors. Other examples include but are not limited to differentialinput transistor pairs, standard gain stages, bandgap configurations.

FIG. 13 depicts a simple example of an operational amplifier with sometypical features of an amplifier included within a largersystem-on-a-chip (SOC). Even this simple example can demonstrate theincreasing complexity of today's verification challenges formixed-signal designs. Feeding the inputs to the amplifier is a simplemultiplexer (mux). The control signal IN_CTRL to the mux will serve as adigital input signal to a series of logic gates within the mux block.These gates will then control the switches to select between two pairsof analog inputs: pair one (IN_POS_1 and IN_NEG_1) and pair 2 (IN_POS_2and IN_NEG_2). For instance, a value of IN_CTRL equal to logic level “0”could close the switches connected to IN_POS_1 and IN_NEG_1 while anIN_CTRL value of logic level “1” could close the switches connected toIN_POS_2 and IN_NEG_2. The analog inputs selected are then output fromthe mux. These signals are now connected to the amplifier inputs. Themux block also has connections for the power supplies. In this casebecause analog signals will pass through the block, the power supplyconnections are to the analog supply voltages AVDD and GND.

For the amplifier block, the power supply connection is also to theanalog supply voltage AVDD and GND. Note that it is not unusual oncomplex SOC's to have multiple internal power supplies. Even for theanalog blocks there may be options such as different power supplylevels. In other cases, the same voltage level may be supplied, but onesupply source may be used for critical blocks that may be susceptible toany noise on the supply line. Other blocks that are noisier (typicallydue to functions such as high frequency switching), may be connected toa different supply source that is the same voltage level. For thisreason, digital blocks are almost always powered separately from analogblocks. In addition, with the increasing importance of reducing power,some power supplies may be turned off during various operating modes. Asa result, design engineers may also need to verify that a given block isconnected to a supply that will be available at the appropriate times.

Another common configuration within a larger chip is to have a commonblock that provides many of the biasing currents required by analogfunctions. In this example, the amplifier has an input for a 1 uA biascurrent. Other common options include signals for trim and control orprogrammability. In this example, the amplifier has 2 control bits,GAIN_CTRL <1:0>, that can set a gain value and 3 bits, GAIN_TRIM<2:0>,that would slightly tune the gain value. Trim is typically performedwhen each device is tested and is used to compensate for manufacturingprocess variations and match the desired gain values to specified valuesas closely as possible. In this case GAIN_CTRL is a programmabilityfeature to pick the best gain setting for a specific application. TheENABLE signal allows the amplifier to be powered on or off while thechip is operating. These types of controls are often provided tominimize power consumption, allow for clean power up sequences of thechip, and to provide protection during fault conditions.

For this simple example, there are numerous examples of verificationrequirements. These requirements can be separated into severalcategories:

-   -   operating function—do the amplifier and mux meet their        respective functional requirements        -   does the mux correctly pass the selected input signals to            the mux output        -   does the amplifier meet it's expected behavior: gain, slew            rate, input range, output range, etc.    -   power supply and biasing        -   is the block connected to the appropriate power supply and            receiving the correct biasing        -   is the supply and biasing available when expected and            operating within the expected range    -   control signals        -   for every setting of control signals, is the correct            behavior observed    -   fault conditions        -   when any of the block pins behaves in an manner outside            allowed or expected ranges, does the block behave            appropriately or is that condition either not possible            (prevented elsewhere) or not expected to be handled

Within the context of this disclosure, one benefit results from moreefficiently managing the process of stepping through many combinationsof possibilities. One of the best examples is just stepping throughlarge numbers of digital control or trim options.

Consider this simple example within the context of a larger SOC and atop-level test bench for this SOC. This test bench may be one of manytest benches for the chip, but this example can show how verificationintellectual property developed for the amplifier can be leveraged evenfrom a top-level test bench. The hierarchical concepts illustrated inFIG. 6 can be populated to show how an example test bench can be built.In this example A1 from FIG. 6 will represent the top-level test benchthat includes the chip, all the necessary circuits or models torepresent the system inputs, output loads, and external power suppliesor other components for the chip, as well as portions of the testsconfiguration that can be described programmatically through software.This example hierarchy can be seen in the diagram in FIG. 6. Asdiscussed previously, each item in the hierarchy can be represented bymultiple types of views depending on the information available and theintended purpose of the block.

The second level of hierarchy in this example that correlates to level“B” in FIG. 6 will include all the elements placed in this top-leveltest bench. The first element will be the chip itself. In this example,the chip is identified as cell B1. B2 could be a cell that containscircuits or models to represent the system inputs to the chip. B3 couldbe a cell that contains circuits or models to represent the systemprovided power supplies and system output loads seen by the chip. B4 inthis example will be a block written in software to connect eitherdirectly to pins of the chip or other blocks, provide values to anyparameterized functions within any of the blocks, and/or monitor andmeasure specific performances during the simulation. In this simplifiedexample, only signals that lead to the creation of the signals in thelower level example amplifier block are included.

This block B4 (or blocks) that can be captured in software willtypically use high-level languages targeted at integrated circuit design(SystemVerilog, Verilog, Verilog-AMS, Verilog-A, SystemC, VHDL, etc.)but can also leverage more general languages and scripts such as C, C++,TCL, PERL, etc. The software portions may create stimulus, define designvalues for parameters in existing circuit or system blocks, or measureand monitor performance throughout the simulation. For example, theUniversal Verification Methodology (UVM) for SystemVerilog includescommon constructs for defining monitoring and scoreboard functions aswell as defining stimulus. Digital verification methodology alsoincludes numerous specific pieces of verification intellectual property(VIP) that are targeted to specific functions or protocols. Forinstanced defined communication standards such as Universal Serial Bus(USB), Serial Peripheral Interface (SPI) bus, etc. will have specificverification IP that can be leveraged during the verification processboth to supply data through the interface and also test and validatethat the interface is matching the protocol standards. In this example,the digital trim and controls will be assumed programmed through a SPIinterface or by the chip itself.

FIG. 14 depicts a simplified example testbench for a SOC. In thisexample, the SOC hierarchy contains the amplifier and mux shown in FIG.13. This example shows how some signals may pass from the top level ofthe design through the hierarchy to a lower level subsystem. In thistestbench, the external power supply VDD_EXT is being generated byVerilog-A code within the TOP_AMST. Then VDD_EXT is an input for theSUPPLIES_AND_OUTPUTS block. That block then outputs VDD which isconnected to the SOC as the primary external power supply. For theinputs, the system inputs are generated within the INPUTS block and thenpassed through the TOP_AMST block before connecting to the SOC. Passingthe inputs through the test harness allows both the voltage and currentof the input signals to be monitored (or used as parameters for othercalculations) throughout the simulation. Similarly VOUT from the SOC ispassed through the TOP_AMST before entering the SUPPLIES_AND_OUTPUTSblock. Finally, the VOUT_EXT is fed from the SUPPLIES_AND_OUTPUTS blockto the TOP_AMST for monitoring. The decisions on which signals should bepassed through the AMST test harness blocks are highly dependent on thesystem and the amount of circuits/models required around the chip toadequately represent the full system. Ultimately these decisions aredriven by the system design and the verification plan.

FIG. 15 depicts a D level example of an electronic design. Signals atlower levels of the hierarchy cannot be accessed unless those signalsare brought to a pin at the top-level. Consider further the hierarchy ofInstance B1 which represents the chip. In this example, the next levelof the chip's hierarchy will contain C1, the SOC_CORE, and C2, theSOC_PAD_RING. Then push down into C1 to see D1, the AMP_CORE subsystem;D2, PWR_MGT (power management functions for the chip); and D3, DIG_CTRL(digital controls for the chip). The AMP_CORE subsystem contains theexample amplifier and mux from FIG. 13.

FIG. 16 depicts a hierarchical example of an electronic design. At theSOC_TESTBENCH, there are four B level portions, B1 SOC (the chip beingverified), B2 INPUTS, B3 SUPPLIES_AND_INPUTS and B4 TOP_AMST. At the Clevel are two portions, C1 SOC_CORE and C2 SOC_PAD_RING. The D levelincludes D1 AMP_CORE, D2 PWR_MGT and D3 DIG_CTRL and the E levelincludes E1 MUX, E2 AMP and E3 AMP_AMST. Throughout this examplehierarchy, instrumentation points could be added.

FIG. 17 depicts an E level example of an electronic design. Furtherdescending into the AMP_CORE subsystem shows the subsystem elements: E1,the MUX; E2, the AMP; and E3, the AMP_AMST which is a VerilogAMS blockcontaining verification IP for this subsystem. AMP_AMST provides aconvenient point to add instrumentation for signals within the AMP_COREhierarchy. Examples include capturing whether all of the amplifiercontrol signals have been exercised and if the inputs to the amplifierare correct based on the inputs to the MUX and have been tested for allpossible combinations of the MUX IN_CTRL. Other information such aswhether the amplifier inputs (AMP_IN_POS, AMP_IN_NEG in FIG. 17) testedby signals supplied to the inputs at the top level of the SOC hierarchy(IN_POS_1, IN_NEG_1, IN_POS_2, IN_NEG_2 in FIG. 14), passed through theSOC_PAD_RING into AMP_CORE (IN_POS_1_CORE, IN_NEG_1_CORE, IN_POS_2_CORE,and IN_NEG_2_CORE in FIG. 15) and passed through the MUX and selected byIN_CTRL fully exercise the input dynamic range of the amplifier. Notethat a verification monitor at the top level of the hierarchy cannotconnect directly to the amplifier inputs, AMP_IN_POS and AMP_IN_NEG. Inthis manner, the instrumentation points in combination with criteriadefined within the AMST, provide a direct measure of verificationcoverage of the amplifier. This data can be calculated cumulatively overnumerous individual simulations and different input conditions.

In the first example which is netlist initiated, FIG. 18, has as itsstarting point a pre-existing netlist of an analog or mixed signaldesign. This flow chart illustrates a computer program product foridentification of useful untested states of an electronic design 1800.The method comprises parsing 1810 at least one netlist of arepresentation of the electronic design comprised at least in part of atleast one analog portion, determining 1812 at least one instrumentationpoint based on the at least one netlist, generating 1814 at least oneinstrumented netlist based on the at least one instrumentation point anddetermining 1816 an analog verification coverage utilizing the at leastone instrumented netlist. The electronic design may be analog or mixedsignal. It is also envisioned that this electronic design may beextended to the verification of electromechanical, electrochemical andelectrobiological systems.

FIG. 19 depicts additional steps that would begin before step one 1812of the first example of instrumentation 1900 of an electronic design. Inthis case the electronic design is the initiation point, from which thenetlist is created and additionally, the specification and theverification history are part of the initial dataset. The example mayadditionally comprise receiving 1910 by a computer a computer readablerepresentation of the electronic design having at least in one part ofthe electronic design, an analog portion and generating 1912 at leastone set of valid states based on the at least one specification. Thefirst example may also comprise receiving 1914 at least one verificationcoverage history of the electronic design, it is envisioned that thecoverage history may be for related circuits, circuits sharing commoncharacteristics and the like. The method may also comprise identifying1916 at least one useful untested state based at least in part upon atleast one of the at least one specification, the at least oneinstrumented netlist, the at least one set of valid states and the atleast one verification coverage history and assessing 1918 verificationcompleteness based at least in part upon at least one of the at leastone instrumented netlist, the at least one verification coverage historyand the at least one set of valid states. The at least one untestedstate are valid states that have not yet been tested. Referring back toFIG. 6, the electronic design would encompass each connected block inthe figure. The top level hierarchy is A1, the next level of hierarchyis B, which includes B1 and B2 and underneath the B hierarchy is the Chierarchy having C1, C2 and C3. In this specific example at the B1 leveland below the model is comprised of Schematic and Spice models. At theB2 level the model encompasses Schematic_behavioral and Behavioral_vamodels. In this instance the analog test model would be connected to A1,instance 1 level and would be connected through the netlist, connectinglines to each of the components under test.

FIG. 20 depicts additional steps that are in addition to the steps inthe FIG. 19 example of instrumentation 2000 of an electronic design, itis envisioned that the steps would fall between step 1914 and 1916, butmay occur at other points in the flow. These additional steps maycomprise correlating 2010 the at least one verification coverage historyand at least one input vector and determining 2012 a minimum number ofthe at least one input vector to maximize the assessed completeness ofverification. The correlation may comprise a measure of sameness,overlap, or the like of input characteristics and assertions. The methodmay further comprise simulating 2014 at a behavioral level of therepresentation of the electronic design the minimum number of the atleast one input vector and assessing 2016 a measurement output at thebehavioral level simulation, wherein the assessment is based at least inpart upon the at least one set of valid states. The method may furthercomprise the steps of simulating 2018 at a transistor level of therepresentation of the electronic design the minimum number of the atleast one input vector and correlating 2020 a measurement output at thetransistor level simulation, wherein the correlation is based at leastin part upon the measurement output at the behavioral level simulation.The method may also comprise receiving 2022 at least one of at least onestimulus and at least one stimulus assertion, receiving 2024 at leastone of at least one output measurement and at least one output assertionand modifying 2026 the at least one instrumented netlist to capture dataindicative of at least one coverage contribution of at least onesimulation. Behavioral level simulations may precede transistor leveland/or behavioral level follow-on tests.

FIG. 21 depicts a second example which is electronic design initiated inthat it begins with a representation of the analog or mixed signalelectronic design. This electronic design initiated method comprises thesteps of receiving 2110 by a computer a computer readable representationof the electronic design having at least in one part of the electronicdesign, an analog portion, generating 2112 at least one instrumentednetlist based at least in part upon the representation of the electronicdesign, receiving 2114 at least one specification of the electronicdesign, generating 2116 at least one set of valid states based on the atleast one specification and determining 2118 an analog verificationcoverage utilizing the at least one instrumented netlist. The untestedstates are valid states that have not yet been tested. Returning back toFIG. 8, the electronic design would encompass each of the connectedblocks following the PMIC_testbench, and at least one analog testharness model would pertain to the block marked PMIC_testbench. Thenetlist for that model would indicate each connection, connecting linein FIG. 8. The hierarchies associated with this design indicate that theLDO, Battery Supervisor and Voltage Reference blocks are lower inhierarchy than the PMIC block. Additionally, this model primarilyreviews the schematic portion of the design.

FIG. 22 depicts additional steps of the electronic design initiatedmethod, it is envisioned that the steps of FIG. 22 would fall after step2112 of FIG. 21, however the order and placement of the steps may vary.The second example of instrumentation 2200 of an electronic design mayadditionally comprise steps such as simulating 2210 the at least oneinstrumented netlist, generating 2212 at least one verification coveragehistory of the electronic design based in part upon the simulation, andidentifying 2214 at least one useful untested state based at least inpart upon at least one of the at least one specification, the at leastone instrumented netlist, the at least one set of valid states and theat least one verification coverage history. The method may furthercomprise assessing 2216 verification completeness based at least in partupon the verification coverage history and the at least one set of validstates, correlating 2218 the at least one verification coverage historyand at least one input vector, wherein the simulation of the at leastone instrumented netlist may be performed at a behavioral level anddetermining 2220 a minimum number of at least one input vector tominimize the useful untested states. The method may also comprisesimulating 2222 at least one of a transistor level and a behaviorallevel of the representation of the electronic design the minimum numberof the at least one input vector, assessing 2224 a measurement output ata behavioral level simulation, wherein the assessment is based at leastin part upon the at least one set of valid states and correlating 2226the measurement output at the transistor level simulation, wherein thecorrelation is based at least in part upon the measurement output at thebehavioral level simulation.

FIG. 23 depicts another electronic design initiated example of acomputer implemented method of instrumentation of an analog or mixedsignal electronic design 2300. This method also includes an assessmentof the verification completeness utilizing the specification,instrumented netlist, a list of valid states and the previousverification coverage that has been performed on the design. The methodcomprises the steps of receiving 2310 by a computer a computer readablerepresentation of the electronic design having at least in one part ofthe electronic design, at least one digital portion and at least oneanalog portion, generating 2312 at least one instrumented netlist basedat least in part upon the representation of the electronic design andreceiving 2314 at least one specification of the electronic design. Themethod also comprises generating 2316 at least one set of valid statesbased on the at least one specification, receiving 2318 at least oneverification coverage history of the electronic design and assessing2320 completeness of verification based at least in part upon at leastone of the at least one specification, the at least one instrumentednetlist, the at least one set of valid states and the at least oneverification coverage history. Referring back to FIG. 9, the electronicdesign would encompass each of the connected blocks following thePMIC_testbench, and at least one analog test harness model would pertainto the block marked PMIC_testbench. The netlist would indicate theconnectivity, shown by the lines connecting each of the blocks. Thehierarchies associated with this design indicate that the LDO, BatterySupervisor and Voltage Reference blocks are lower in hierarchy than thePMIC block. This model reviews the schematic, schematic behavioral andbehavioral.va aspects of the model. Though the overall electronic designis the same as in FIG. 8, what is tested, the hierarchies andconnectivity are different.

FIG. 24 depicts additional steps that may occur after step 2318 of FIG.23, although the exact point of inclusion may alter depending upon thespecific implementation. The third example of instrumentation 2400 of anelectronic design may additionally comprise such as correlating 2410 theat least one verification coverage history and at least one inputvector, determining 2412 a minimum number of the at least one inputvector to maximize the assessing completeness of verification andsimulating 2414 at a behavioral level of the representation of theelectronic design the minimum number of the at least one input vector.The method may also comprise assessing 2416 a measurement output at thebehavioral level simulation, wherein the assessment is based at least inpart upon the at least one set of valid states, simulating 2418 at atransistor level of the representation of the electronic design theminimum number of the at least one input vector and correlating 2420 themeasurement output at the transistor level simulation, wherein thecorrelation is based at least in part upon the measurement output at thebehavioral level simulation.

In a fourth example, FIG. 25 illustrates an electronic design initiatedbehavioral model. The computer program product 2500 is embodied on anon-transitory computer usable medium 2510. The non-transitory computerusable medium having stored thereon a sequence of instructions which,when executed by at least one processor 2512 causes the at least oneprocessor to execute a method of identification of useful untestedstates of an electronic design, comprising the steps of receiving 2514 arepresentation of the electronic design comprised at least in part of atleast one analog portion and generating 2516 at least one instrumentednetlist based at least in part upon the representation of the electronicdesign. The computer program product further comprises the steps ofreceiving 2518 at least one specification of the electronic design,generating 2520 at least one set of valid states based on the at leastone specification and simulating 2522 at a behavioral level of therepresentation of the electronic design the at least one instrumentednetlist at a minimum number of the at least one input vector. Thecomputer program product further comprises the steps of generating 2524at least one verification coverage history of the electronic designbased in part upon the simulation and identifying 2526 useful untestedstates based at least in part upon at least one of the at least onespecification, the at least one instrumented netlist, the at least oneset of valid states and the at least one verification coverage history.Referring back to FIG. 10, the electronic design would encompass each ofthe connected blocks following the PMIC_testbench, and at least oneanalog test harness model would pertain to the block markedPMIC_testbench. In this example the PMIC hierarchy and the LDO, BatterySupervisor and Voltage Reference hierarchies are reviewed. At thehighest level the PMIC is reviewed at the schematic level and the LDO atthe Behavioral_vams, the Battery Supervisor at the Behavioral_vams andthe Voltage reference at the Behavioral_va level.

FIG. 26 depicts additional steps beginning at some point after step 2516of FIG. 25. The steps may additionally comprise assessing 2610 ameasurement output at a behavioral level simulation, wherein theassessment is based at least in part upon the at least one set of validstates, correlating 2612 the at least one verification coverage historyand at least one input vector and simulating 2614 at a transistor levelof the representation of the electronic design the minimum number of theat least one input vector. The example may also comprise correlating2616 the measurement output at a transistor level simulation, whereinthe correlation is based at least in part upon the measurement output atthe behavioral level simulation, receiving 2618 at least one outputmeasurement and receiving at least one output assertion and receiving2620 at least one analog test harness model.

FIG. 27 depicts a post-simulation aggregated analysis as a fifth exampleof a computer implemented method of instrumentation of an electronicdesign 2700. In this case the simulations have been run and a databaseof information collected which allows a review looking for holes in theverification coverage. The analysis method comprises the steps ofreceiving 2710 a representation of the electronic design, having atleast in one part of the electronic design, an analog portion, receiving2712 at least one specification of the electronic design, generating2714 at least one set of valid states delimited by the at least onespecification, receiving 2016 at least one verification history of theelectronic design and identifying 2718 useful untested states based atleast in part upon at least one of the at least one specification, theat least one set of valid states and the at least one verificationhistory. Referring back to FIG. 8, the electronic design would encompasseach of the connected blocks following the PMIC_testbench, and at leastone analog test harness model would pertain to the block markedPMIC_testbench. The netlist for that model would indicate eachconnection, connecting line in FIG. 8. The hierarchies associated withthis design indicate that the LDO, Battery Supervisor and VoltageReference blocks are lower in hierarchy than the PMIC block.Additionally, this model primarily reviews the schematic portion of thedesign.

FIG. 28 depicts additional steps that may occur at any point after step2710 of FIG. 27. In this example, the data inputs may vary and allow abroader review of the states that have been reviewed and what additionalstates need to be reviewed. This fifth example of instrumentation 2800of an electronic design may additionally comprise steps such asreceiving 2810 at least one manufacturing process variation of theelectronic design, receiving 2812 at least one functional variation ofthe electronic design wherein the identifying useful untested statesbased at least in part upon the at least one manufacturing processvariation and the at least one functional variation and receiving 2814at least one of at least one stimulus and at least one stimulusassertion. The method may also comprise receiving 2816 at least one ofat least one output measurement and at least one output assertion,receiving 2818 at least one analog test harness model, analyzing 2820 atleast one output of a simulation and identifying 2822 verification statecoverage based at least in part upon at least one of the at least oneverification history and at least one level of abstraction of the atleast one specification. Referring back to FIG. 9, the electronic designwould encompass each of the connected blocks following thePMIC_testbench, and at least one analog test harness model would pertainto the block marked PMIC_testbench. The netlist would indicate theconnectivity, shown by the lines connecting each of the blocks. Thehierarchies associated with this design indicate that the LDO, BatterySupervisor and Voltage Reference blocks are lower in hierarchy than thePMIC block. This model reviews the schematic, schematic behavioral andbehavioral.va aspects of the model. Though the overall electronic designis the same as in FIG. 8, what is tested, the hierarchies andconnectivity are different.

In a sixth example, FIG. 29 illustrates a manufacturing inclusive reviewof the verification coverage of an analog or mixed signal design. Inthis example not only the specification is taken into account, but alsomanufacturing process variations that will likely be encountered andprobable functional variations encountered post manufacturing. In thisexample the computer program product 2900 is embodied on anon-transitory computer usable medium 2910, the non-transitory computerusable medium having stored thereon a sequence of instructions which,when executed by a processor 2912 causes the processor to execute amethod of identification of useful untested states of an electronicdesign, comprising the steps of, receiving 2914 a representation of theelectronic design comprised at least in part of at least one analogportion, receiving 2916 at least one specification of the electronicdesign, receiving 2918 at least one manufacturing process variation ofthe at least one analog portion of the electronic design and receiving2920 at least one functional variation of the at least one analogportion of the electronic design. The computer program product alsocomprises the steps of generating 2922 at least one set of valid statesdelimited by one of the at least one specification, the at least onemanufacturing process variation and the at least one functionalvariation, receiving 2924 at least one verification history of the atleast one analog portion of the electronic design and identifying 2926useful untested states based at least in part upon at least one of theat least one specification, the at least one set of valid states, the atleast one manufacturing process variation, the at least one functionalvariation and the at least one verification history. The product mayalso include the step of assessing a verification completeness based atleast in part upon the at least one verification history and the atleast one set of valid states.

FIG. 30 illustrates an example system diagram 3000 for identification ofuseful untested states of an electronic design. A user's schematiccapture module 3010 captures a representation of the electronic design.A netlist is generated 3012 from the captured schematic and the netlistis processed by a netlist processor 3014. The netlist is subsequentlyinstrumented 3016 to allow inputs and outputs at netlist nodes to beindependently captured and a simulation is performed at a transistorlevel by a transistor level simulator 3018. Data from the simulator isstored in a simulator output database 3020. The simulation output datais post processed by a simulation post processor 3022, which feeds thedata into a project level hierarchical tested states database 3024. Acoverage analysis and model checking module 3026 receives data from theproject level hierarchical tested states database. The data from thesimulator output database may also be sent to the user's schematiccapture and/or waveform viewer 3028 which feeds data to the designdatabase 3030. The design database may also send and receive data fromthe project level hierarchical tested states database.

Although the examples have been directed to analog electronic testing,the methods, system and computer readable media may be applied toelectro-mechanical, electro-chemical and electro-biological verificationand testing.

While the making and using of various exemplary examples of thedisclosure are discussed herein, it is to be appreciated that thepresent disclosure provides concepts which can be described in a widevariety of specific contexts. Although the disclosure has been shown anddescribed with respect to a certain example, it is obvious thatequivalents and modifications will occur to others skilled in the artupon the reading and understanding of the specification. The presentdisclosure includes such equivalents and modifications, and is limitedonly by the scope of the following claims.

It is to be understood that the method and apparatus may be practicedlocally, distributed or remotely and that the data for steps may bestored either locally or remotely. For purposes of clarity, detaileddescriptions of functions, components, and systems familiar to thoseskilled in the applicable arts are not included. The methods andapparatus of the disclosure provide one or more advantages includingwhich are not limited to, improved speed efficiency, decreasedcomputation time, decreased number of re-verifications and the like.While the disclosure has been described with reference to certainillustrative examples, those described herein are not intended to beconstrued in a limiting sense. For example, variations or combinationsof steps in the examples shown and described may be used in particularcases while not departing from the disclosure. Various modifications andcombinations of the illustrative examples as well as other advantagesand examples will be apparent to persons skilled in the arts uponreference to the drawings, description, and claims.

1. A computer implemented method of instrumentation of an electronicdesign, comprising: receiving by a computer a computer readablerepresentation of said electronic design having at least in one part ofsaid electronic design, an analog portion; generating at least oneinstrumented netlist based at least in part upon said representation ofsaid electronic design; receiving at least one specification of saidelectronic design; generating at least one set of valid states based onsaid at least one specification; and determining an analog verificationcoverage utilizing said at least one instrumented netlist.
 2. Thecomputer implemented method of instrumentation of said electronic designof claim 1, further comprising: simulating said at least oneinstrumented netlist; generating at least one verification coveragehistory of said electronic design based in part upon said simulation;and identifying at least one useful untested state based at least inpart upon at least one of said at least one specification, said at leastone instrumented netlist, said at least one set of valid states and saidat least one verification coverage history.
 3. The computer implementedmethod of instrumentation of said electronic design of claim 2, furthercomprising assessing verification completeness based at least in partupon said verification coverage history and said at least one set ofvalid states.
 4. The computer implemented method of instrumentation ofsaid electronic design of claim 3, further comprising correlating saidat least one verification coverage history and at least one inputvector.
 5. The computer implemented method of instrumentation of saidelectronic design of claim 3, wherein said simulation of said at leastone instrumented netlist is performed at a behavioral level.
 6. Thecomputer implemented method of instrumentation of said electronic designof claim 5, further comprising determining a minimum number of at leastone input vector to minimize said useful untested states.
 7. Thecomputer implemented method of instrumentation of said electronic designof claim 6, further comprising simulating at least one of a transistorlevel and a behavioral level of said representation of said electronicdesign said minimum number of said at least one input vector.
 8. Thecomputer implemented method of instrumentation of said electronic designof claim 7, further comprising assessing a measurement output at abehavioral level simulation, wherein said assessment is based at leastin part upon said at least one set of valid states.
 9. The computerimplemented method of instrumentation of said electronic design of claim8, further comprising correlating said measurement output at saidtransistor level simulation, wherein said correlation is based at leastin part upon said measurement output at said behavioral levelsimulation.
 10. A computer program product embodied on a non-transitorycomputer usable medium, said non-transitory computer usable mediumhaving stored thereon a sequence of instructions which, when executed byat least one processor causes said at least one processor to execute amethod of instrumentation of an electronic design, comprising: receivingby a computer a computer readable representation of said electronicdesign having at least in one part of said electronic design, an analogportion; generating at least one instrumented netlist based at least inpart upon said representation of said electronic design; receiving atleast one specification of said electronic design; generating at leastone set of valid states based on said at least one specification; anddetermining an analog verification coverage utilizing said at least oneinstrumented netlist.
 11. The computer program product to execute saidmethod of method of instrumentation of said electronic design of claim1, further comprising: simulating said at least one instrumentednetlist; generating at least one verification coverage history of saidelectronic design based in part upon said simulation; and identifying atleast one useful untested state based at least in part upon at least oneof said at least one specification, said at least one instrumentednetlist, said at least one set of valid states and said at least oneverification coverage history.
 12. The computer program product toexecute said method of method of instrumentation of said electronicdesign of claim 11, further comprising the step of assessingverification completeness based at least in part upon said verificationcoverage history and said at least one set of valid states.
 13. Thecomputer program product to execute said method of method ofinstrumentation of said electronic design of claim 12, furthercomprising the step of correlating said at least one verificationcoverage history and at least one input vector.
 14. The computer programproduct to execute said method of method of instrumentation of saidelectronic design of claim 13, wherein said simulation of said at leastone instrumented netlist is performed at a behavioral level.
 15. Thecomputer program product to execute said method of method ofinstrumentation of said electronic design of claim 14, furthercomprising the step of determining a minimum number of at least oneinput vector to minimize said useful untested states.
 16. The computerprogram product to execute said method of method of instrumentation ofsaid electronic design of claim 15, further comprising the step ofsimulating at least one of a transistor level and a behavioral level ofsaid representation of said electronic design said minimum number ofsaid at least one input vector.
 17. The computer program product toexecute said method of method of instrumentation of said electronicdesign of claim 16, further comprising the step of assessing ameasurement output at a behavioral level simulation, wherein saidassessment is based at least in part upon said at least one set of validstates.
 18. The computer program product to execute said method ofmethod of instrumentation of said electronic design of claim 17, furthercomprising the step of correlating said measurement output at saidtransistor level simulation, wherein said correlation is based at leastin part upon said measurement output at said behavioral levelsimulation.